1. Field of the Invention and Related Art Statement
The present invention relates to a system test apparatus for automatically verifying whether or not a system works according to particular specifications.
Conventionally, when a new product is developed, a verification is conducted to determine whether or not the product's operating system works according to its design specifications.
When such a verification is conducted, test items which comprise setting data and expected values (expected information) to be output from the system in response to the given setting data are generated along a time axis. By providing the setting data to the system, the system is actually operated. After that, the verification is conducted by comparing the detected operation information of the system with the expected information along the time axis.
For example, if the system to be verified is a refrigerator, assume the following design specifications:
"When the main switch is pressed, the motor which rotates the compressor is started.
"When the voltage of the temperature sensor which measures the temperature becomes 3.12 V, the motor is stopped."
"When the voltage of the temperature sensor which measures the temperature becomes 1.57 V, the LED is lit up."
First, according to the test items which are prepared based on the design specifications, the setting data, corresponding to the main switch and the temperature sensor respectively, are generated along the time axis. Also, the expected values according to the motor and the LED are determined.
Second, the setting data is input to the refrigerator successively. The resulting the operation information of the refrigerator is detected and compared with the expected information along the time axis.
For example, in the first step, the terminal voltage of the main switch is initialized to 5 V as the setting data. Then, in the second step, the terminal voltage of the main switch is changed to 0 V thereby creating the state in which the main switch is pressed.
When the operation information obtained indicates that the motor at rest during the first step is started in the second step, the verification of the design specification "when the main switch is pressed, the motor which rotates the compressor is started" is complete.
As described above, the setting data is conventionally input to the system in sequence. Verification of whether or not the system works according to the design specifications is effected by comparing the detected operation information with the expected operation information along the time axis.
In such verification steps, due to errors involved in the detecting devices and those of sensors in the system, some information cannot be strictly defined with respect to an exact time.
For example, in a system which deals with temperature, such as a refrigerator, a state where "the temperature becomes -21.5.degree. C." is detected by a sensor. If the sensor has an error of .+-.0.5.degree. C., it is possible that the above state took place at any time when the temperature is between -21.0.degree. C. and -22.0.degree. C. That is, the time at which such state took place cannot be determined with precision.
In addition, if the system specifications have a redundancy, the system state may not be uniquely determined.
Thus, when such a system verification is automatically conducted, it is important to deal with the information containing such errors and redundancy. To date, techniques for dealing with errors and redundancies have not been achieved and automatic verification is not easily accomplished.
In the verification method described above, the verification items are created by a person using his or her knowledge and experience due to time restrictions associated with the verification. Thus, occasionally the verification items are insufficient and thereby the system tests are not satisfactorily conducted. Consequently, some errors in the system may not be found. Most of such errors are attributable to abnormal processes of the product which were not fully tested in the test phase of the product. The later such errors are found, the more expensive the repair cost becomes. Thus, it is necessary to evaluate whether or not an abnormal process is fully tested.